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Technical  Objectives: 


The  Objectives  of  this  project  are  to  design  a  distributed  network  environment  for  infrared 
sensor  image  acquisition  and  signal  processing  and  implement  the  design  in  Rome 
Laboratory’s  internal  network.  The  technical  problem  is  to  manipulate  12  bit  digital  infrared 
image  data  in  an  efficient  manner  that  exploits  all  of  the  computing  power  available  on  the 
network.  The  General  methodology  Is  to  maintain  the  network  operating  systems  at  the 
release  levels  that  will  optimize  the  computational  throughput  of  the  network,  and  to 
arrange  the  data  so  that  it  is  logically  close  to  the  user  and  network  node  that  will  perform 
the  mathematical  operations.  The  proximate  location  of  the  data  and  the  compute  engine 
reduces  the  effect  of  network  bandwidth  limitations  inherent  in  conventional  Ethernet 
networks.  This  arrangement  of  data  will  improve  the  effective  performance  of  the  individual 
nodes  on  the  network. 

Report  Summary  and  Technical  Results: 


The  implementation  of  a  distributed  network  environment  for  infrared  sensor  image 
acquisition  and  signal  processing  was  performed  in  three  phases.  First,  the  available 
network  resources  were  upgraded,  second  the  data  acquisition  process  was  improved  and 
third,  the  network  was  reconfigured  to  reduce  data  flow  bottlenecks.  This  report  will  detail 
the  process  by  which  the  three  phases  were  implemented  and  summarize  the  current 
configuration  of  network  resources. 

The  first  phase  of  the  program  was  to  perform  a  thorough  evaluation  of  the  current 
workstation  capabilities  and  to  determine  if  the  hardware  is  suitable  for  processing  the 
data  from  the  next  generation  of  infrared  cameras.  In  this  environment,  it  is  crucial  to 
evaluate  the  workstation  products  of  the  major  manufacturers  not  only  in  terms  of  which 
company  has  the  lead  in  price/performance  today,  but  which  computing  systems  have 
the  most  potential  for  growth  when  specifically  considering  image  processing  of  infrared 
data. 

While  it  is  tempting  to  form  the  network  by  including  hardware  from  all  of  the  major 
manufacturers,  thereby  taking  advantage  of  each  of  their  strengths,  this  approach 
presents  a  software  nightmare.  Separate  versions  of  all  source  code  and  executable 
binary  code  must  be  stored  and  made  available  to  users,  in  a  heterogeneous 
computing  environment.  In  addition  to  multiplying  the  required  disk  space  by  several 
factors,  such  a  network  would  surely  have  a  higher  system  traffic  and  would  require 
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many  times  the  development  and  maintenance  effort  to  resolve  source  code 
incompatibilities  between  platforms. 

The  open  architecture  of  the  SparcStations  makes  them  ideal  for  data  acquisition  from 
either  high  speed  digital  tape  recorders,  or  custom  camera  frame  buffers,  consequently, 
it  was  determined  that  a  homogeneous  SUN  workstation  environment  would  be  most 
efficient.  In  addition,  we  decided  to  upgrade  all  the  workstations  in  the  network  to  the 
latest  stable  version  of  the  operating  system,  SUNOS  4.1.3.  This  permitted  the  use  of  some 
of  the  slower  workstations  as  diskless  nodes  that  functioned  as  display  servers.  In  general, 
it  is  found  that  90%  of  the  computing  work  on  the  network  is  performed  by  the  top  three 
fastest  workstations. 

During  this  period,  several  hard  disk  failures  occurred  which  required  hardware 
replacement,  an  8mm  streaming  tape  drive  with  a  5  GB  formatted  capacity  was  installed, 
several  Sbus  frame  buffer  boards  were  installed,  and  work  was  begun  on  the  Sbus 
implementation  of  the  DR11W  I/O  interface  which  was  currently  functioning  on  one  of  the 
slower  workstations. 

In  addition  a  yellow  pages  database  was  created  with  all  the  user  information.  The 
database  would  permit  quick  reconfiguration  of  network  resources  in  the  event  of  a 
workstation  crash  or  network  reorganization.  The  users  home  directories  were  centralized 
onto  three  servers  for  simpler  administration  and  backup.  The  8mm  tape  drive  was 
configured  to  backup  all  user  files  from  these  servers  every  night  at  1:15  AM.  A  minimum  of 
one  month  of  backups  is  maintained  which  enables  users  to  recover  lost  files  provided  they 
discover  the  loss  within  30  days. 

The  second  phase  of  the  program  targeted  the  data  acquisition  requirements  of  Rome 
Laboratory,  but  continued  to  modify  network  resource  as  needed.  The  data  acquisition 
capabilities  of  the  current  workstations  were  studied  and  optimized  to  capture  infrared 
data.  Clearly  there  is  no  best  approach  to  capture  data  from  an  infrared  camera; 
different  types  of  measurements  have  different  data  acquisition  requirements. 
Nevertheless,  the  chosen  data  acquisition  mechanisms  can  accommodate  virtually  all 
of  the  measurement  requirements. 

Three  major  data  acquisition  items  were  completed  during  this  phase  of  the  program.  First, 
a  faster,  more  compact  and  more  reliable  Sbus  version  of  the  VME  based  DR11W  I/O 
board,  called  the  S11W  was  installed  in  a  SparcStation  10/30.  This  board  and  workstation 
combination  reduced  the  frame  transfer  time  from  the  camera  to  the  network  by  several 
orders  of  magnitude.  The  transfer  time  required  for  a  320  x  244  camera  operating  at  30 
frames  per  second  is  approximately  0.25  s. 
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The  second  data  acquisition  item  that  was  completed  was  the  selection  and  purchase  of  a 
SPARC  based  laptop  computer  for  use  in  mobile  frame  acquisition.,  and  the  reconfiguration 
of  an  additional  SPARC  based  laptop  for  archiving  acquired  data  frames  to  an  8mm  tape. 
The  laptop  that  initially  arrived  from  the  manufacturer  was  not  functional  due  to  a  memory 
parity  error  which  created  random  load  errors  during  the  installation  of  the  operating 
system.  The  fault  was  diagnosed  and  sent  back  to  the  manufacturer  for  repair.  The  repaired 
laptop  was  configured  as  the  SparcStation  10/30  for  frame  acquisition  using  the  SI  1W  I/O 
board. 

The  third  data  acquisition  item  that  was  completed  was  the  evaluation  and  purchase  of 
portable  hard  disk  drives.  Three  2.4  GB  hard  disks  were  installed  on  the  network. 
Performance  benchmarks  indicated  that  each  hard  drive  will  hold  450  seconds  of  data  from 
a  320  X  244  camera  operating  at  30  frames  per  second.  The  peak  transfer  rates  for  these 
hard  disks  is  10  MB/sec,  and  the  sustained  transfer  rate  for  these  hard  disks  is  1 .9  MB/sec. 

The  SparcStation  10/30,  and  both  of  the  laptops  were  configured  to  automatically  sense 
and  mount  whichever  of  the  three  hard  disk  drives  were  attached.  In  addition  a  backup 
mechanism  was  implemented  which  permitted  all  of  the  data  in  each  of  the  hard  disks  to  be 
backed  up  to  an  8mm  tape  with  a  single  command.  These  modifications  were  performed  for 
a  joint  Israel  and  USA  data  acquisition  measuremeht  program. 

The  latest  trends  in  workstation  networking  indicate  that  UNIX  Vr4  will  be  the  future 
operating  system  standard.  Currently  the  RL  network  is  using  SUNOS  UNIX  based  on 
BSD4.3.  Migration  of  Rome  Laboratory’s  network  to  the  UNIX  Vr4  standard  will 
increase  their  real  time  data  acquisition  capabilities,  but  will  also  improve  their  network 
performance.  UNIX  Vr4  will  have  a  more  robust  Network  File  System  (NFS), 
automounter,  and  Network  Information  Service  (NIS).  In  addition,  NFS  data  throughput 
is  expected  to  improve  by  moving  to  streams  from  socket  based  Remote  Procedure 
Calls  (RPC).  The  newest  version  of  UNIX  Vr4  from  Sun  Microsystems  is  expected  to  be 
capable  of  fully  symmetric  multiprocessing  and  multithreading.  These  are  tremendous 
advantages  to  signal  processing  problems  on  arrays  of  data  which  can  be  easily 
vectorized. 

During  this  phase  several  improvements  and  modifications  to  the  network  were 
implemented,  including  the  evaluation  of  the  Solaris  2.1,  2.2,  and  2.3  versions  of  the 
operating  system  which  is  based  on  UNIX  Vr4.  Only  the  Solaris  2.3  version  of  the  operating 
system  was  found  to  be  stable  enough  for  limited  use.  The  Solaris  2.X  versions  of  the 
operating  system  are  attractive  alternatives  to  SUNOS  because  the  support  multi-processor 
based  workstations. 
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In  addition  to  operating  system  changes,  a  C  compiler  for  Solaris  2.3  was  installed  on  the 
network,  a  128  gray  level  printer/plotter  was  installed  for  image  display,  and  a  SparcStation 
10/512  and  SparcStation  10/40  was  installed  on  the  network,  and  an  additional  8mm  tape 
drive  was  installed  for  the  Israel  measurements.  The  SparcStation  10/512  is  a  two  CPU 
workstation  with  1MB  of  secondary  static  RAM  cache  for  each  processor.  Several 
additional  frame  acquisition  hardware  tools  were  also  installed  on  the  network  during  this 
time. 

The  third  phase  of  the  program  attempted  to  reconfigure  the  network  to  reduce  data  flow 
bottlenecks.  Achieving  the  necessary  performance  from  these  workstations  depends  on 
balancing  network  performance  with  individual  computer  system  performance.  For 
example,  the  speed  with  which  a  workstation  can  capture  an  image  is  important,  but  if 
the  image  is  to  be  displayed  remotely,  across  an  Ethernet  connection,  the  ability  to 
display  a  sequence  of  frames  to  form  a  video  may  be  limited  by  the  network  bandwidth. 

The  network  architecture  which  links  the  various  workstations  was  evaluated  and 
reconfigured  to  maximize  data  throughput.  Without  considering  alternatives  to  the 
Ethernet  standard,  an  improvement  in  network  performance  was  achieved  with  a  study 
of  the  data  paths  throughout  the  network,  and  a  subsequent  targeting  of  the  data  flow 
bottlenecks  which  were  identified.  Specifically,  the  location  of  data  archives,  data 
acquisition,  user  files,  image  processing  computation,  and  processed  image  display 
were  reorganized  to  eliminate  bottlenecks  in  the  data  flow. 

The  network  model  developed  to  optimize  the  data  flow  is  consistent  with  the  three  file 
server  approach  implemented  in  the  first  phase  of  this  program  if  the  file  servers  are 
also  acting  as  compute  servers.  This  model  is  not  consistent  with  typical  client/server 
implementations. 

Typically  a  client/server  application  is  structured  to  emphasize  data  services  on  the 
server  and  compute  services  on  the  client.  The  intent  in  client/server  applications  is  to 
take  advantage  of  many  low  cost  processors  available  on  the  clients  to  distribute  the 
compute  load,  while  maintaining  the  data  integrity  and  administration  functionality  of 
the  a  mainframe  (with  reduced  compute  requirements),  on  the  server. 

This  approach  works  well  when  the  data  sets  are  small,  and  the  compute  requirements 
on  the  data  can  be  easily  partitioned  into  independent  tasks.  The  infrared  image 
processing  problem  deals  with  data  sets  that  are  huge  and  compute  requirements  that 
can  be  partitioned  into  dependent  tasks.  We  have  converged  on  a  model  which 
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reduces  the  importance  of  network  bandwidth  by  combining  the  image  processing 
compute  services  and  the  data  services  onto  the  server  and  off-loading  the  display 
compute  services  onto  the  client. 

This  model  has  room  to  grow  with  the  application  of  multiprocessor  based  computing  to 
the  server. 
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Cost  Elements; 
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